No data, good data

One of the things I particularly loved about my Pebble Time was how every firmware update brought something new and exciting. The Timeline made sense and, after having spent just a while with a colleague’s Pebble Steel, felt even more like the right way to go forward. One of the things that made the Timeline make sense was how scrolling through past or future events was rather efortless, intuitively mapping to the “up” and “down” buttons.

The 4.0 firmware update however, changed Timeline in a way that quickly polarized Pebble users. The change, apparently minor, consisted in reassigning the “up” button from Timeline past to Pebble Health while also – most probably unintentionally – preventing access to past events altogether. Combined with a rather eager way of marking pins and events as belonging to the past this has annoyed a lot of users that used to consult past or all-day events on their watch. Consider the following scenario: the Weather app adds pins for sunrise and sunset by default. Besides providing useful info on sunrise/sunset times, they also acted as a shortcut for the day’s forecast, right in the Timeline. With the new update however, once the clock was past sunrise, gone were both the day’s forecast and all all-day events.

On the official forums and on twitter, some users raged about what they considered an ill-informed design decision at best or a nefarious push of the Pebble Health app down the user’s… wrists. Of course, other users, and even some Pebble staff, suggested that the Timeline past was a seldomly used feature and that a lot more people where actually using Pebble Health. And as such the update isn’t that big of a deal.

About a week later a firmware update was issued that brought Timeline past back as a shortcut option for the any of the Pebble’s 4 physical buttons.

Looking back at how the events around the coming and going of the “Timeline past” and the different kinds of data used to explain the initial decisions, I couldn’t help but remember an anecdote about how the British Air Force went about improving the armor on their bombers. As bombers and bomber pilots were highly prized assets of the British Air Force, efforts were made to ensure as many of them as possible would get home safely after bombing raids over the Reich’s positions. Initially, the people studying the problem looked at the returning bombers, analyzed the damage the enemy flak inflicted and decided to increase the armor on the areas most often hit by flak. However, the situation didn’t improve that much. It wasn’t until Abraham Wald, a mathematician, made an insightful observation that things radically improved. His observation? All the bombers that have returned actually survived, despite the damage taken from enemy anti-aircraft defenses, so the armor should be improved on the un-damaged areas. The planes that got hit in crucial spots, did not return home to tell their story. Afterwards, armor around the cockpit and other delicate bits was improved and, as they say, the rest is history. (You can read more about the story, with a link to the original paper here).

What does this have to do with Pebble and the removal of a seemingly unused feature?

Let me explain. If you own a Pebble and decide not to use Pebble Health and some other services you have a rather “silent” smartwatch as far as exposing your data to third parties goes. As soon as you want to use Pebble Health or apps that use the Dictation API, you have to allow Pebble, to, among others, collect usage data. What this means, is that through the very design of the usage reporting metrics, people using Pebble Health were over-represented, while people that didn’t were, most likely, missing from said metrics because they didn’t enable usage reporting – either explicitly or by not bothering to enable them (Pebble were nice enough to have this disabled by default).

So, in a way, Pebble was right that Timeline past was seldomly used – that’s what the data said. What they overlooked however, what what the data did not say.